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2 PRIORITY 

3 The present application claims priority of Provisional 

4 Application, Application Nximber 60/117,866, having the 

5 same title and a filing date of 01/29/99, by inventors 

6 Gordon Braudaway et al , 

7 CROSS REFERENCES 

8 The present application is related to the following 

9 applications even dated with the above referenced 

10 Provisional Application: Application Number 09/240,242, 

11 entitled, "Recovering Invisible Digital Image 

12 Watermarks From Distorted Images," by inventors Gordon 

13 Braudaway et al*; and Application Number 60/117,921 

14 entitled, "Watermarking and Determining Distortion in 

15 an Image," by inventors Gordon Braudaway et al., which 

16 are incorporated herein by reference in their entirety. 
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1 FIELD OF THE INVENTION 

2 This application relates to the field of digitized 

3 imaging. It is more specifically directed to 

4 determining and removal of image distortion from an 

5 image . 

6 BACKGROUND OF THE INVENTION 

7 With the development of means of production and 

8 circulation of digital images, and the means of 

9 imbedding relatively invisible watermarks into digital 

10 images ostensibly to convey ownership of the image, 

11 there is now financial incentive to attack an imbedded 

12 watermark in an attempt to render it non-extractable. 

13 Pixel locations of a watermarked image are presumed to 

14 correspond to those in an unmarked original image. 

15 Generally, the watermark is imbedded by altering only 

16 the values of the pixel components of the original 

17 image, not their geometric positions. This may be 

18 accomplished employing such methods as described in US 

19 Patent 5,825,892 which is incorporated herein by 

20 reference in its entirety. 

21 Some methods of attacking an imbedded watermark rely on 

22 constructing a new image that is a geometrically 
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1 distorted copy of the watermarked image. This new 

2 image is herein referred to as a distorted copy, 

3 Pixels in the distorted copy are placed at subtly 

4 distorted positions relative to those in the 

5 watermarked image. Pixel component values in the 

6 distorted copy are determined by two-dimensional 

7 interpolation of component values of enclosing pixel in 

8 the watermarked image* No constraints can be placed on 

9 the types of pixel position distortion an attacker 

10 might choose to use. To those skilled in the art, 

11 however, it is obvious that excessive pixel 

12 position-distortion will cause the distorted copy to be 

13 a caricature of the watermarked image, thus diminishing 

14 or destroying its economic value. Whether a distortion 

15 is excessive is a subjective measure. For a distorted 

16 copy to be useful, it requires that linear or nonlinear 

17 distortion methods that are used by an attacker have to 

18 be used sparingly and in such a manner as to produce 

19 smoothly varying and relatively small position 

20 distortions. This is so as to be essentially 

21 unobjectionable and casually unnoticeable to untrained 

22 observers. The human visual system, as a qualitative 

23 measuring device, can be relied upon to readily detect 

24 excessive distortion. It is desirable to have a method 

25 of defense that requires little or no limits to be 

26 placed on pixel position-distortions produced by the 

27 attacking method. 



3 

DOCKET NUMBER; YO999-034 



1 



T 



1 SUMMARY OF THE INVENTION 

2 In one aspect, the present invention provides a method, 

3 apparatus and article of manufacture employing an 

4 undistorted reference image relative to which 

5 measurements of distortion are made. These employ an 

6 automatic method for composing a realigned image which 

7 does not depend on a process of visually examining a 

8 composite image, and recording the coordinates of pixel 

9 locations closest to common image features in a 

10 distorted image and reference image. If the presumed 

11 distorted image is not the same size as the reference 

12 image, it is made so by shrinking or enlarging the 

13 reference image. At the next step, at least three 

Lii 14 distinct points that do not form a straight line are 

15 selected. The integer coordinates of the pixels 

16 nearest these points are herein referred to as 

17 reference centers. At each reference center, a segment 
^'1 18 of each image, herein referred to as a sub -image, is 

O 19 excised, thus producing for each image as many 

20 sub -images as there are distinct points. The 

21 horizontal and vertical dimensions of the sub -images, 

22 in integer pixel coordinates, are respectively based on 

23 the common horizontal and vertical dimensions of the 

24 images. A pair of sub -image, one excised from the 

25 reference image and the other excised from the 

26 distorted image, having the same reference center are 

27 manipulated to determine and/or substantially remove 

28 the distortion from the distorted image. 



4 

DOCKET NUMBER; YO999-034 



1 Another aspect of the present invention enables a 

2 digital image watermark to be extracted from a 

3 geometrically distorted copy of a reference image, 

4 Other aspects and a better understanding of the 

5 invention may be realized by referring to the Detailed 

6 Description. 

7 BRIEF DESCRIPTION OF THE DRAWING 

8 These and other objects, features, and advantages of 

9 the present invention will become apparent upon further 

10 consideration of the following detailed description of 

11 the invention when read in conjunction with the drawing 

12 figure • 

13 DESCRIPTION OF THE INVENTION 

14 The present invention provides a method, apparatus and 

15 article of manufacture by which the distortion in a 

16 distorted copy of an image is automatically measured 

17 and removed sufficiently well that a relatively 

18 invisible image watermark extraction is possible • In 

19 an embodiment of the invention, the method employs a 

20 reference image relative to which measurements of 

21 distortion are made. For example, an original unmarked 

22 image or a relatively invisibly watermarked copy of the 
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1 original image, called a watermarked image, serve 

2 equally well as the reference image. 

3 The automatic method composes a realigned image in a 

4 way that does not depend on a process of visually 

5 examining a composite image and recording the 

6 coordinates of pixel locations closest to common image 

7 features in a distorted copy and reference image, as is 

8 the case in the semiautomatic method of Application 

9 Number 09/240,242, cross-referenced herewith. 

10 As with the semiautomatic method/ if the presumed 

11 distorted copy is not the same size as the reference 

12 image, it is made so by shrinking or enlarging the 

13 distorted copy using a method such as pixel 

14 interpolation or pixel extrapolation. The distorted 

15 copy, after being geometrically resized, is herein 

16 referred to as a geometric alteration. There are well 

17 known automatic and manual methods for resizing images. 

18 At the next step, the automatic process differs from 

19 the semiautomatic process. At least three common pixel 

20 locations in the two images, herein referred to as 

21 reference centers, not lying in any straight line are 

22 required for the automatic method of this invention. 

23 In an example embodiment, a coarse but regularly spaced 

24 simulated mesh is superimposed identically on each of 

25 the two images . Each knot in the mesh forms one of the 

26 common pixel locations . The integer coordinates of the 

27 pixels nearest each of the at least three common pixel 
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1 locations are the reference centers. At each reference 

2 center, a segment of each image, herein referred to as 

3 a sub -image, is excised, thus producing for each image 

4 as many sub -images as there are common pixel locations. 

5 The horizontal and vertical dimensions of the 

6 sub -images, in integer pixel coordinates, are based on 

7 the common horizontal and vertical dimensions, 

8 respectively, of the images. The pair of sub -images, 

9 one excised from the reference image and the other 

10 excised from the geometric alteration, having the same 

11 reference center are herein referred to as 

12 corresponding sub -images. A typical embodiment employs 

13 three common pixel locations • 

14 When the geometric alteration is laid upon the 

15 reference image, the intersection of pixels in the 

16 reference image with pixels in the geometric alteration 

17 is an area of each image called the safe area. 

18 Although initially the safe area is the entire area of 

19 the geometric alteration and the reference image, on 

20 subsequent iterations of this realignment method the 

21 safe area may be smaller than the reference image. 

22 Each sub -image is centered at its respective reference 

23 center. If any pixel of a sub -image extends beyond the 

24 boundaries of the safe area, such as at the edges of 

25 the safe area, then the top or bottom edge of the 

26 corresponding sub -images, and the left or right edge of 

27 the corresponding sub -images, if also necessary, are 

28 repositioned. The two subimages, in unison, are 

29 repositioned vertically and horizontally by the minimum 
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1 number of pixel locations necessary so that no pixel of 

2 either sub -image lies in whole or in part outside of 

3 the safe area. The corresponding reference center is 

4 adjusted to lie at the center of the repositioned 

5 sub -images. Reference centers, after this adjustment 

6 process, are referred to as adjusted centers, whether 

7 they were adjusted or not. Note that in the case where 

8 the sub -image horizontal width, J, and vertical height, 

9 J, are even integers, such as powers of two, and the 

10 coordinates of the sub -images are indexed 0 to J-1 and 

11 0 to tJ-l, respectively, the integer coordinates of the 

12 center of each sub -image are defined to be 1/2 and J/ 2. 

13 Also note that I should be less than the width and J 

14 should be less than the height of the reference image 

15 and the geometric alteration. 

16 The n-th pair of sub -images, one from the reference 

17 image designated the n-th reference sub -image and a 

18 corresponding one from the geometric alteration 

19 designated the n-th distorted sub- image, is used to 

20 compute a two-dimensional cross -correlation surface 

21 relating the distorted sub -image with the reference 

22 sub -image image. Those skilled in the art will 

23 recognize that there are many methods that may be used 

24 to compute or approximate a cross -correlation surface 

25 relating the two corresponding sub -images. Regardless 

26 of the method used, any method that produces the 

27 intended result of determining the horizontal and 

28 vertical offsets of the distorted sub -image relative to 

29 the reference sub -image that achieves a good match may 
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1 be employed. For example, a good match has offsets 

2 that can not be improved by more than the spacing of 

3 1.5 pixels in any direction by using any other means of 

4 pattern matching. A better match is readily achievable 

5 if the distortions are generally linear. Watermark 

6 detection seldom requires a match better than a 0.5-2.0 

7 pixel spacing. If required, more complex iteration 

8 techniques may be used to achieve a best match. 

9 In an example embodiment, forward and inverse discrete 

10 Fast Fourier Transforms (FFT's) are used to compute a 

11 cross -correlation surface. Note that for methods using 

12 FFT*s, all sub- images are converted to monochrome, if 

13 not already so. The non- integer interpolated 

14 horizontal and vertical offsets of the peak of the 

15 cross -correlation surface, Pn and qn, relative to the 

16 origin of the cross -correlation surface are used as 

17 additive offsets relating the center of the n-th 

18 distorted sub -image relative to the n-th corresponding 

19 adjusted center. Thus, for each pair of sub -images, 

20 the coordinates of the adjusted center are the center 

21 on the n-th reference sub -image. The siims of the 

22 coordinates of the adjusted center plus the offsets of 

23 the peak of the n-th cross -correlation surface become 

24 the approximate coordinates of the center of the 

25 distorted sub image, Un = Xn+Pn and Vn = ^n+gn. The 

26 coordinates of the offset center of the distorted 

27 sub -sub image and the adjusted center, herein referred 

28 to as computed pixel coordinates, are analogous to the 

29 measured pixel coordinates of common features visually 
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1 selected from the geometric alteration and reference 

2 image of the previously cited semiautomatic method. 

3 Using the computed pixel coordinates instead of the 

4 measured pixel coordinates, the coefficients matrix. A, 

5 of the pixel position interpolation equations are 

6 computed, as in the semiautomatic method. 

7 In another example embodiment, forward and inverse 

8 discrete Fast Fourier Transforms (FFT's) are used to 

9 compute a modified cross -correlation surface. The 

10 magnitudes of the Fourier transform coefficients are 

11 modified to make said magnitudes uniform. An inverse 

12 Fourier transform is used to compute the modified 

13 cross -correlation surface. 

14 In yet another example embodiment, a weighted sum of 

15 the ordinary and modified Fourier transform 

16 coefficients is formed before using an inverse Fourier 

17 transform to compute a weighted cross correlation 

18 surface. 

19 In an embodiment of the automatic method, use of a 

20 statistical screen is incorporated into the next step. 

21 Use of a statistical screen is generally not needed in 

22 the semiautomatic method. Because the selection of 

23 sub -images is done indiscriminately, it is possible 

24 that some sub -images may have few features (or even 

25 none) . The cross -correlation surfaces computed from 

26 such sub -images may be relatively flat and have a 

27 misleading peak. An additional test is used to 
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1 discriminate against statistical out- flyers that can 

2 occur from such misleading peaks. As used herein, an 

3 'out -flyer' is a value from a set of values that 

4 deviates so greatly from the other values in the set 

5 that it is statistically unlikely to be a member of the 

6 set. 

7 An example statistical screen discriminating against 

8 out -flyers is embodied as follows. Each of the pairs 

9 of computed pixel coordinates, Un and Vn, is processed 

10 by the interpolation equations to give a proposed 

11 reference center, Xn' and yn' . The Euclidean distance 

12 between the proposed reference center and the 

13 corresponding adjusted center, Xn and gn/ is computed 

14 for each of the n sub -images. Out -flyers are deleted 

15 from the set of n Euclidean distances, largest first, 

16 based on their value being above a first threshold 

17 value. If any out -flyer is deleted, the offset center 

18 of its corresponding distorted sub -image and adjusted 

19 center are also deleted. The coefficients, A, are 

20 recomputed using the undeleted pairs of pixel 

21 coordinated, but never with fewer than three pairs of 

22 pixel coordinates. 

23 From this point on, the automatic and semiautomatic 

24 methods generally parallel one another. The square 

25 submatrix, Sa/ of four of the coefficients of the 

26 interpolation equations is factored into four primitive 

27 image manipulation matrices. The geometric alteration 
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1 is manipulated by the four primitive image 

2 manipulations to form the reoriented image. 

3 The entire restoration process is advantageously 

4 repeated iteratively by substituting the previously 

5 reoriented image for the geometric alteration before 

6 each subsequent iteration. For the automatic method, 

7 it has been found that for images with significant 

8 distortion, as many as three iterations are warranted. 

9 For attacks with more nonlinear distortion, additional 

10 iterations may be used to further improve reorientation 

11 of the geometric alteration. The iteration process is 

12 terminated when examination of the Euclidean distances 

13 shows that the reduction of all Euclidean distances 

14 relative to those from the previous iteration is less 

15 than a second threshold. Finally, the reoriented image 

16 is realigned left or right and up or down relative to 

17 the reference image, based on the offset coefficients c 

18 and f from the interpolation equations, to form the 

19 realigned image. 

20 An overview of the steps of an example automatic 

21 realignment of a presumed distorted copy is shown in 

22 the figure. First, if the presumed distorted copy is 

23 not the same size as the reference image, it is shrunk 

24 or enlarged to make it the same size using any image 

25 resizing method; the resized distorted copy is herein 

26 referred to as the geometric alteration, (102) . 
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1 In a particular einbodiment , a common coarse but 

2 regularly spaced mesh having n knots is superimposed on 

3 both of the images, (104) . The knots of the mesh serve 

4 as reference points for each of the images, A list of 

5 the horizontal and vertical coordinates, and ifn, of 

6 pixels lying closest to the knots of the mesh is built; 

7 these coordinate pairs are herein referred to as the 

8 reference centers, (106) . A sub -image, having 

9 dimensions J and J and centered at each of the 

10 reference centers, is excised from each of the images 

11 and converted to monochrome, if not already so, (108) , 

12 producing n corresponding pairs of sub images. A safe 

13 area is determined as the intersection of pixels common 

14 to the geometric alteration and reference image (110) . 

15 If any part of any sub- image lies beyond the boundaries 

16 of the safe area, it is repositioned left or right and 

17 up or down a minimum number of pixel locations until 

18 both it and its corresponding sub -image lie within the 

19 boundaries of the safe area, and their common reference 

20 center is adjusted to again lie at the center of the 

21 repositioned sub -images, (112) . A two-dimensional 

22 cross correlation surface is computed from each of the 

23 n pairs of sub -images, (114) . The non- integer 

24 horizontal and vertical coordinate offsets, Pn and Qn, 

25 of the greatest peak on each of the n cross -correlation 

26 surfaces are determined by two-dimensional 

27 interpolation, (116) . The distorted reference centers 

28 are computed by adding the coordinate offsets to their 

29 corresponding adjusted centers, (118) . Using the 

30 adjusted centers and the distorted reference centers in 
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1 stead of the manually measured pixel coordinate pairs, 

2 as in the referenced semiautomatic method, the 

3 coefficients matrix. A, of the pixel position 

4 interpolation equations is computed, (120) , in a manner 

5 identical to that used in the cross-referenced 

6 semiautomatic method. 

7 In this embodiment, the automatic method differs from 

8 the cross-referenced semiautomatic method in at least 

9 the next steps. Using the pixel position interpolation 

10 equations, each of the distorted reference centers is 

11 converted to form a proposed reference center. The 

12 Euclidean distance between each of the proposed 

13 reference center and its corresponding adjusted center 

14 is computed. Those that are "out-f lyers" , for example 

15 those greater than a first threshold, say a spacing of 

16 5 pixels, are discarded while at least three are always 

17 retained, (122) . If any of the distorted reference 

18 centers is discarded, (124) , steps (120) through (124) 

19 are repeated using only the not -discarded adjusted 

20 centers and the not -discarded distorted reference 

21 centers. Otherwise, as in the referenced semiautomatic 

22 method, the square submatrix, Sk, of four of the 

23 coefficients is factored into four primitive image 

24 manipulating matrices, (126), and the geometric 

25 alteration is manipulated by the four primitive image 

26 manipulations to form the reoriented image, (128) . 

27 If any of the primitive manipulations is not 

28 sufficiently small, as determined by comparing the 
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1 incremental changes of the not -discarded Euclidean 

2 distances to a second threshold, (130) , the geometric 

3 alteration is replaced by the reoriented image, (132) , 

4 and steps (104) through (130) are repeated. Otherwise, 

5 the reoriented image is offset according to the 

6 computed horizontal and vertical offset values to form 

7 the realigned image, (134) . Often, an attempt is made 

8 to extract the invisible watermark from the realigned 

9 image, (136) . 

10 It is noted that the foregoing has outlined some of the 

11 more pertinent objects and embodiments of the present 

12 invention. This invention may be used for many image 

13 or image- like applications. Thus, although the 

14 description is made for particular arrangements and 

15 methods, the intent and concept of the invention is 

16 suitable and applicable to other arrangements and 

17 applications. It will be clear to those skilled in the 

18 art that modifications to the disclosed embodiments can 

19 be effected without departing from the spirit and scope 

20 of the invention. The described embodiments ought to 

21 be construed to be merely illustrative of some of the 

22 more prominent features and applications of the 

23 invention. Other beneficial results can be realized by 

24 applying the disclosed invention in a different manner 

25 or modifying the invention in ways known to those 

26 familiar with the art. 
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CLAIMS : 



2 Having thus described our invention, what we claim as 

3 new and desire to secure by Letters Patent is as 

4 follows: 

5 1. A method for restoring a geometrically distorted 

6 copy of a reference image, said method comprising: 

7 automatically determining a type and amount of 

8 distortion of said distorted copy; and 

9 substantially reversing the distortion to form a 

10 reoriented image. 

11 2, A method as recited in claim 1, further comprising 

12 horizontally and vertically aligning the reoriented 

13 image with the reference image to form a realigned 

14 image . 

15 3. A method as recited in claim 2, further comprising 

16 extracting a watermark from the realigned image. 

17 4. A method as recited in claim 1, wherein the step of 

18 automatically determining includes: 

19 composing a geometric alteration of the distorted copy 

20 by making the distorted copy the same size as the 

21 reference image; 
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1 defining a safe area having safe pixels, wherein said 

2 safe area is an intersection of pixels in the reference 

3 image with pixels in the geometric alteration, and said 

4 safe pixels includes any pixel from the reference image 

5 or the geometric alteration which lies in the safe 

6 area; 

7 selecting 'n' points in the safe area, wherein 'n' is 

8 at least three and not all 'n' points lie on a straight 

9 line; 

10 building a list of 'n' reference centers, wherein each 

11 reference center corresponds to coordinates of a 

12 particular pixel lying closest to a particular one of 

13 said 'n' points; 

14 constituting a plurality of pairs of sub -images, 

15 wherein each pair is centered at one of said reference 

16 centers and each pair is formed by a sub -image from the 

17 geometric alteration and a corresponding sub -image from 

18 the reference image; 

19 minimally horizontally and vertically positioning from 

20 an original sub -image position to a new sub -image 

21 position any sub -image pair having any sub -image pixel 

22 lying outside the safe area, such that said any 

23 sub -image pixel lies within the safe area; 

24 adjusting the reference center of said any sub -image 

25 pair to correspond to said new sub -image position; 
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1 computing a two-dimensional cross correlation surface 

2 from each of the 'n' pairs; 

3 locating a horizontal, Pn, and a vertical, g^, 

4 coordinate offset of the greatest peak on each 

5 cross -correlation surface; 

6 calculating a plurality of distorted centers; 

7 using the adjusted centers and the corresponding 

8 distorted centers to compute the coefficients matrix, 

9 A, of a pixel position interpolation equation; and 

10 forming a sub -matrix, Sa# from the first, second, 

11 fourth and fifth elements of the matrix A, 

12 5* A method as recited in claim 4, further comprising: 

13 computing a set of proposed reference centers based on 

14 the distorted reference centers and the pixel position 

15 interpolation equations; 

16 computing the Euclidean distances between the proposed 

17 reference centers and the adjusted centers; 

18 testing each Euclidean distances to determine if said 

19 each Euclidean distance is statistically improbable; 
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1 discarding said each distorted center and its 

2 corresponding adjusted center that define a Euclidean 

3 distance that is statistically improbable while 

4 retaining at least three not-discarded distorted 

5 centers and their corresponding adjusted centers; and 

6 recomputing the coefficients matrix. A, of a pixel 

7 position interpolation equation using the at least 

8 three not -discarded distorted centers and corresponding 

9 adjusted centers. 

10 5. A method as recited in claim 4, wherein the steps 

11 of substantially reversing includes: 

12 factoring the sub -matrix Sa into four primitive image 

13 manipulation matrices; 

14 applying the four primitive image manipulation matrices 

15 to the geometric alteration to produce a reoriented 

16 image; 

17 making the geometric alteration identical to the 

18 reoriented image if any primitive image manipulation 

19 matrix produces a distortion greater than a 

20 predetermined threshold, said distortion determined by 

21 examining all iteration- to- iteration incremental 

22 changes in the Euclidean distances; and 

23 repeating all the steps of claim 4 except the step of 

24 composing, all the steps of claim 5, and the steps of 
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1 factoring, applying and making until no primitive image 

2 manipulation matrix produces a distortion greater than 

3 the predetermined threshold. 

4 ?• A method, as recited in claim 4, wherein the step 

5 of horizontally and vertically aligning includes 

6 translating horizontally the reoriented image by the 

7 value of the third coefficient of the matrix. A, and 

8 translating vertically the reoriented image by the 

9 value of the sixth coefficient of the matrix. A, to 

10 form the realigned image. 

11 8. A method as recited in claim 3, wherein the step of 

12 composing includes: 

13 shrinking or enlarging the distorted copy vertically by 

14 pixel interpolation or extrapolation such that the 

15 produced geometric alteration has a same height as the 

16 reference image, and 

17 shrinking or enlarging the produced geometric 

18 alteration horizontally by pixel interpolation or 

19 extrapolation to have the same width as the reference 

20 image. 

21 9. A method as recited in claim 4, wherein the 

22 coordinate offsets are non- integers and the step of 

23 locating includes using interpolation. 
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1 10. A method, as recited in claim 1, wherein the 

2 reference image is an original unmarked image. 

3 11. A method as recited in claim 4, wherein the step 

4 of computing includes: 

5 comparing a region in the geometric alteration 

6 surrounding each of said reference centers with regions 

7 in the reference image shifted in position by a 

8 multiplicity of coordinate offsets; 

9 ascertaining horizontal and vertical coordinate offsets 

10 of each selected reference center as being the 

11 horizontal and vertical offset at which the region in 

12 the geometric alteration and the region on the 

13 reference image most nearly match; and 

14 calculating the distorted centers from the coordinate 

15 offsets and the corresponding reference centers. 

16 12 . A method as recited in claim 11, wherein the step 

17 of comparing includes: 

18 using a modified cross correlation function in which 

19 the Fourier transform of the cross correlation function 

20 is computed/ wherein magnitudes of the Fourier 

21 transform coefficients are modified to make said 

22 magnitudes uniform, and 
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1 using an inverse Fourier transform to compute the 

2 modified correlation function. 

3 13. A method as recited in claim 11, wherein the step 

4 of comparing includes: 

5 using a weighted cross correlation function in which 

6 the Fourier transform of the cross correlation function 

7 is computed/ wherein a weighted sum of the ordinary and 

8 modified Fourier transform coefficients is formed, and 

9 using an inverse Fourier transform to compute the 

10 weighted correlation function. 

11 14. A method as recited in claim 8, wherein the step 
H 12 of ascertaining includes: 

. 

13 composing a geometric alteration of the distorted copy 

O 14 by making the distorted copy the same size as the 

15 reference image; 



16 defining a safe area having safe pixels, wherein said 

17 safe area is an intersection of pixels in the reference 

18 image with pixels in the geometric alteration, and said 

19 safe pixels includes any pixel from the reference image 

20 or the geometric alteration which lies in the safe 

21 area; 

22 building a list of reference centers, wherein each 

23 reference center corresponds to coordinates of a 
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1 particular pixel in the safe area lying closest to a 

2 particular one of said at least three pixel locations; 

3 computing a plurality of corresponding distorted 

4 centers; 

5 using the reference centers and the corresponding 

6 distorted centers to compute coefficients of a matrix, 

7 A, of pixel position interpolation equations; 

8 testing each distorted center to determine if said each 

9 distorted center is statistically improbable; and 

10 discarding each distorted center that is statistically 

11 improbable while retaining at least three not-discarded 

12 distorted centers, until no more distorted centers are 

13 discarded. 

14 15. An article of manufacture comprising a computer 

15 usable medium having computer readable program code 

16 means embodied therein for causing a restoring of a 

17 geometrically distorted copy of a reference image, the 

18 computer readable program code means in said article of 

19 manufacture comprising computer readable program code 

20 means for causing a computer to effect: 

21 automatically determining a type and amount of 

22 distortion of said distorted copy; and 
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1 substantially reversing the distortion to form a 

2 reoriented image. 

3 16, An article of manufacture as recited in claim 15, 

4 the computer readable program code means in said 

5 article of manufacture further comprising computer 

6 readable program code means for causing a computer to 

7 effect horizontally and vertically aligning the 

8 reoriented image with the reference image to form a 

9 realigned image. 

10 17. An article of manufacture as recited in claim 16, 

11 the computer readable program code means in said 

12 article of manufacture further comprising computer 

13 readable program code means for causing a computer to 

14 effect extracting a watermark from the realigned image 

15 18. An article of manufacture as recited in claim 15, 

16 wherein the reference image is a copy of an original 

17 unmarked image. 

18 19. An article of manufacture as recited in claim 15, 

19 wherein the step of automatically determining includes 

20 selecting a set of at least three reference centers in 

21 the geometric alteration; 

22 comparing a region in the geometric alteration 

23 surrounding each of said reference centers, with 
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1 regions in the reference image shifted in position by a 

2 multiplicity of coordinate offsets; 

3 ascertaining horizontal and vertical coordinate offsets 

4 of each selected reference centers as being the 

5 horizontal and vertical offset at which the region in 

6 the geometric alteration and the region on the 

7 reference image most nearly match; and 

8 determining the type and amount of distortion from the 

9 coordinate offsets and the set of reference centers. 

10 20, An article of manufacture as recited in claim 19, 

11 wherein the step of comparing includes: 

12 using a modified correlation function in which the 

13 Fourier transform of the correlation function is 

14 computed, wherein the magnitudes of the Fourier 

15 transform coefficients are modified to make said 

16 magnitudes uniform, and 

17 using an inverse Fourier transform to compute the 

18 modified correlation function. 

19 21, An article of manufacture as recited in claim 19, 

20 wherein the step of ascertaining includes performing a 

21 least -squares fit on the horizontal and vertical 

22 coordinate offsets of the set of reference centers. 
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1 22. A computer program product comprising a computer 

2 usable medium having computer readable program code 

3 means embodied therein for causing a readjusting of a 

4 geometrically distorted copy of a reference image, the 

5 computer readable program code means in said computer 

6 program product comprising computer readable program 

7 code means for causing a computer to effect: 

8 automatically determining a type and amount of 

9 distortion of said distorted copy; and 

10 substantially reversing the distortion to form a 

11 reoriented image. 

12 23. An apparatus for restoring a geometrically 

13 distorted copy of a reference image, said method 

14 comprising: 

15 means for automatically determining a type and amount 

16 of distortion of said distorted copy; and 

17 means for substantially reversing the distortion to 

18 form a reoriented image. 

19 24. An apparatus as recited in claim 23, further 

20 comprising horizontally and vertically aligning the 

21 reoriented image with the reference image to form a 

22 realigned image. 
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1 25. An apparatus as recited in claim 24, further 

2 comprising means for extracting a watermark from the 

3 realigned image. 

4 26. An apparatus as recited in claim 23, wherein the 

5 means for automatically determining includes: 

6 means for composing a geometric alteration of the 

7 distorted copy by making the distorted copy the same 

8 size as the reference image; 

9 means for defining a safe area having safe pixels, 

10 wherein said safe area is an intersection of pixels in 

11 the reference image with pixels in the geometric 

12 alteration, and said safe pixels includes any pixel 

13 from the reference image or the geometric alteration 

14 which lies in the safe area; 

15 means for selecting 'n' points in the safe area, 

16 wherein 'n' is at least three and not all 'n' points 

17 lie on a straight line; 

18 means for building a list of 'n' reference centers, 

19 wherein each reference center corresponds to 

20 coordinates of a particular pixel lying closest to a 

21 particular one of said 'n' points; 

22 means for constituting a plurality of pairs of 

23 sub -images, wherein each pair is centered at one of 

24 said reference centers and each pair is formed by a 
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1 sub -image from the geometric alteration and a 

2 corresponding sub -image from the reference image; 

3 means for minimally horizontally and vertically 

4 positioning from an original sub -image position to a 

5 new sub -image position any sub -image pair having any 

6 sub -image pixel lying outside the safe area, such that 

7 said any sub -image pixel lies within the safe area; 

8 means for adjusting the reference center of said any 

9 sub -image pair to correspond to said new sub -image 

10 position; 

11 means for computing a two-dimensional cross correlation 

12 surface from each of the 'n' pairs; 

13 means for locating a horizontal, Pn, and a vertical, 

14 qn, coordinate offset of the greatest peak on each 

15 cross -correlation surface; 

16 means for calculating a plurality of distorted centers; 

17 means for using the adjusted centers and the 

18 corresponding distorted centers to compute the 

19 coefficients matrix. A, of a pixel position 

20 interpolation equation; and 

21 means for forming a sub -matrix, Sa/ from the first, 

22 second, fourth and fifth elements of the matrix A. 
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1 27. An apparatus as recited in claim 26, further 

2 comprising: 

3 means for computing a set of proposed reference centers 

4 based on the distorted reference centers and the pixel 

5 position interpolation equations; 

6 means for computing the Euclidean distances between the 

7 proposed reference centers and the adjusted centers; 

8 means for testing each Euclidean distances to determine 

9 if said each Euclidean distance is statistically 

10 improbable; 

11 means for discarding said each distorted center and its 

12 corresponding adjusted center that define a Euclidean 

13 distance that is statistically improbable while 

14 retaining at least three not -discarded distorted 

15 centers and their corresponding adjusted centers; and 

16 means for recomputing the coefficients matrix. A, of a 

17 pixel position interpolation equation using the at 

18 least three not -discarded distorted centers and 

19 corresponding adjusted centers. 

20 28. An apparatus as recited in claim 26, wherein the 

21 coordinate offsets are non- integers, and the means for 

22 locating includes using interpolation. 
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1 29. A method as recited in claim 1, wherein the steps 

2 of automatically determining and substantially 

3 reversing are repeatedly applied until an amount of the 

4 distortion falls below a given threshold. 

5 30. A method as recited in claim 29, wherein the given 

6 threshold is less than a 0.5 pixel spacing. 

7 31. A method comprising: 

8 automatically measuring the degree of 

9 distortion imparted upon a distorted replica 

10 of an original image; 

11 substantially reversing the degree of 

12 distortion of the distorted replica to form 

13 an undistorted image; and 

14 aligning the undistorted image with the 

15 original image. 

16 32. A method as recited in claim 31, further 

17 comprising determining whether the distorted replica 

18 image has been geometrically distorted relative to 

19 another form of the original image. 

20 33. An article of manufacture comprising a computer 

21 usable medium having computer readable program code 

22 means embodied therein for causing restoration of an 

23 image, the computer readable program code means in said 
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1 article of manufacture comprising computer readable 

2 program code means for causing a computer to effect: 

3 automatically measuring the degree of 

4 distortion imparted upon a distorted replica 

5 of an original image; 

6 substantially reversing the degree of 

7 distortion of the distorted replica to form 

8 an undistorted image; and 

9 aligning the undistorted image with the 

10 original image, 

11 34. An article of manufacture as recited in claim 32, 

12 the computer readable program code means in said 

13 article of manufacture further comprising computer 

14 readable program code means for causing a computer to 

15 effect determining whether the distorted replica image 

16 has been geometrically distorted relative to another 

17 form of the original image. 

18 35. A method as recited in claim 4, wherein the step 

19 of selecting 'n' points includes employing a simulated 

20 mesh with knots. 
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1 



COMPOSING A REALIGNED IMAGE 



2 ABSTRACT OF THE INVENTION 

3 This invention provides methods, apparatus and article 

4 of manufacture used as a count ermeasure to image 

5 distorting. The present invention involves 

6 automatically detecting the presence of distortion in a 

7 presumed distorted image, measuring the magnitude and 

8 type of distortion, and finally creating a realigned 

9 image. Once image distortion is removed, conventional 

10 invisible watermark extraction methods are employed to 

11 extract the watermark from the realigned image. The 

12 automatic method does not depend on a process of 

13 visually examining a composite image and recording the 

14 coordinates of pixel locations closest to common image 

15 features in a distorted image and reference image. 

16 Generally, the presumed distorted image is resized to 

17 the same size as the reference image. Reference 

18 centers at at least three distinct points that do not 

19 form a straight line are selected. At each reference 

20 center, a sub -image is excised. Pairs of sub -images, 

21 one excised from the reference image and the other 

22 excised from the distorted image, having the same 

23 reference center are used in the process of removal of 

24 the distortion. 
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Make image, presumed to be a distorted image, the same size 
as the reference image, and call it a geometric alteration. 



Superimpose a common coarse, but regularly spaced, mesh 
having n knots on the images 



Build a list of reference centers, Xn and j^, which are the 
coordinates of the n pixel lying closest to the knots. 



Excise a sub-image, centered at each reference center, from 
each image and convert each sub-image to monochrome. 



Determine a safe area from the intersection of pixels in the 
geometric alteration with those in the reference image. 



I 



Reposition any sub-image pair lying totally or partially outside 
the safe area to be inside, and adjust the reference centers. 



Compute a two-dimensional cross-correlation surface from each 
pair of sub-images 



Locate, by interpolation, horizontal and vertical offsets, and 
Qn, of the fundamental peak in each cross-correlation surface. 



Compute distorted reference centers as 

Un = Xn + Pn and v„ =y« + 




1 


Using adjusted centers and corresponding distorted reference 
centers, compute coefficients. A, of the interpolation equation. 




^^ 


Test each distorted reference center and discard those that are 
statistically improbable, retaining at least three. 






A 


Were any discarded? ||^Yes ' 



No 



1 



Factor Sa into four primitive image manipulation matrices 



Apply the primitive image manipulations to the geometric 
alteration to produce a reoriented image. 





Were all primitive 
image manipulations 
sufficiently small? 


No,^ 


Replace geometric 
alteration with the 
reoriented image 




y 


Yes 

f 








Offset the reoriented image horizontally and vertically to form 
the realigned image. 





Attempt to extract the invisible image watermark from the 
realigned image. 
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